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THE CLAIMS 

1 . A method of processing a digital image, each pixel of the digital image 
having only a single sampled value, the method comprising interpolating values of a 
first color at pixels where the first color was not sampled, the interpolation of the first 
color value at a given pixel including: 

determining likelihoods of the given pixel belonging to the same region as 
each of at least two other pixels having sampled values of the first color, the other 
pixels in different directions relative to the given pixel; and 

using the likelihoods and the sampled values of the other pixels to interpolate 
the first color at the given pixel. 

2. The method of claim 1 , wherein at least two of the directions are 
orthogonal. 

3. The method of claim 1 , wherein the directions include north, south, east 
and west of the given pixel. 

4. The method of claim 1 , wherein the sampled pixel values are of 
neighboring pixels nearest the given pixel. 

5. The method of claim 1 , further comprising additionally using sampled 
values of a second color to compute terms for correcting the sampled values of the 
first color. 

6. The method of claim 5, wherein using the sampled values of the second 
color to correct a sampled value of the first color includes taking a difference between 
the sampled value at the given pixel and the sampled value of the second color at a 
neighbor, the neighbor lying in the same direction as the pixel being corrected. 
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7. The method of claim 6, wherein the likelihoods are used to compute a 
weighted average of the sampled values and correction terms. 

8. The method of claim 1 , wherein determining the likelihoods includes 
applying a similarity function to differences between sampled values. 

9. The method of claim 8, wherein determining the likelihoods includes 
using the differences to index a lookup table of precomputed likelihoods. 

10. The method of claim 1 , wherein the likelihoods are used to compute a 
weighted average of the sampled values. 

1 1 . The method of claim 1 , wherein the first color is green, whereby missing 
information in a green color plane is interpolated. 

12. The method of claim 11, wherein missing green information (I G ) at a 
given pixel (i 9 j ) is computed as 

£ {Jl[m(i, j) - m(i + 2aJ + 2b)} + m(i + aj + b)}w a b (i, j) 

where m(ij) is a sampled pixel value, a and b are components of a vector (a,b), X is 
a tunable parameter, TV is a neighborhood of pixels, and w a b are weights 
corresponding to the vector (a,b). 

13. The method of claim 12, wherein the weights w a b are computed as 

™ a jb J) = Pi m # + la > J + 2b ^ ~ m ^ f>\ + \ m< j + ^ 7 + *) - m (i -*J- b)\] , where pi] is a 
similarity function. 

14. The method of claim 1 , wherein the digital image corresponds to a 
Bayer CFA. 
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15. The method of claim 1 , further comprising interpolating at least one 
other color at each pixel. 

16. The method of claim 15, wherein the likelihoods are used to interpolate 
missing information in one color plane, and wherein bilinear interpolation is used to 
interpolate missing information in other color planes, wherein the interpolation 
includes using sampled and interpolated green pixel values. 

1 7. The method of claim 1 5, further comprising interpolating additional 
missing values, the interpolation of a second color value at the given including: 

using sampled and interpolated first color values to determine likelihoods of 
the given pixel belonging to the same region as neighboring pixels; and 

using the likelihoods and sampled second color values in the neighborhood to 
interpolate the second color value at the given pixel. 

18. The method of claim 17, further comprising computing correction terms; 
and using the correction terms to correct the sampled second color values in the 
neighborhood. 

19. The method of claim 18, wherein a missing color value 7 c (i,j) at pixel 
(i.j) is computed as 

X Uc (U j) - 1 c (i + aj + b) + m(i + a, j + bj\ u ab (i, j) 

IA*>J) = ^ — 

where c e {R,B}, and 
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{(-1,-1), , {(1-1), (1,1)} , 



{(0,-l),(0,l)}, 



{(-1,0), (1,0)}, 



c — R, i is odd, j is even, or 

c = B, i is even, j is odd 
c = B, i is odd, j is even, or 

c = R, i is even, j is odd 
c = R, i is odd, j is odd, or 

c = B, i is even, j is even 



and 



u 



a b (/, j) = p\l G 0, j) - I c (i + a, j + b)\] . 



20. The method of claim 1 5, wherein interpolating at least one of the other 
missing colors includes: 

making an initial estimate for the other missing colors; 

transforming an output image into a luminance-chrominance color space, the 
output image including sampled and interpolated values of the fist color, and sampled 
values of the other colors; 

smoothing the luminance and chrominance bands; 

transforming the output image back to its original color space; and 

resetting measured values and green interpolated values in the output image 

21 . The method of claim 20, wherein each step is a linear operation, and 
wherein the steps are performed by applying a concatenation of the linear operations 
to the demosaic image. 

22. The method of claim 1 5, wherein affine interpolation of at least one of 
the other colors is performed as 



where l d is the color to be interpolated at pixel coordinates (x,y), l c is either the green 
sampled value or the green interpolated value, s diC is a slowly varying function of a 
shift in color c, and td, c is a slowly varying function that can change the intensity 
gradient. 



i d (*> y) = t d * y)i c (*> y) + s d, c (X y) 
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23. Apparatus comprising a processor for interpolating missing values in a 
first color plane of a mosaic image, the interpolation of a given pixel in the color plane 
including: 

determining likelihoods of neighbors belonging to the same region as the 
given pixel, the neighbors having sampled values of the first color, at least two of the 
neighbors lying in different directions relative to the given pixel; and 

using the likelihoods and the sampled values to interpolate a first color value 
for the given pixel. 

24. The apparatus of claim 23, wherein the processor also uses sampled 
values in a second color plane to compute correction terms for the sampled values in 
the first color plane. 

25. The apparatus of claim 24, wherein using the sampled values in the 
second color plane includes taking a difference between the sampled value at the 
given pixel and a sampled value in the second color plane at a neighbor, the 
neighbor lying in the same direction as the sampled value being corrected. 

26. The apparatus of claim 25, wherein the likelihoods are used to compute 
a weighted average of the sampled values and the correction terms. 

27. The apparatus of claim 23, wherein determining the likelihoods includes 
applying a similarity function to differences between sampled values. 

28. The apparatus of claim 23, wherein the first color is green, whereby 
missing information in a green color plane is interpolated. 

29. The apparatus of claim 23, wherein the processor also interpolates at 
least one other missing color value at each pixel. 
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30. The apparatus of claim 23, wherein the processor further interpolates a 
second color plane, the interpolation of a second color value at the given pixel 
including: 

using sampled and interpolated first color values to determine likelihoods of 
the given pixel belonging to the same region as neighboring pixels; and 

using the likelihoods and sampled second color values in the neighborhood to 
interpolate the second color value at the given pixel. 

31 . The apparatus of claim 23, wherein the processor further interpolates 
additional color planes, the interpolation of the additional color planes including: 

making an initial estimate for missing information in the additional color planes; 

transforming an output image into a luminance-chrominance color space, the 
output image including sampled and interpolated values of the fist color, and sampled 
values of the other colors; 

smoothing the luminance and chrominance bands; 

transforming the output image back to its original color space; and 

resetting measured values and green interpolated values in the output image. 

32. The apparatus of claim 23, wherein the processor further interpolates a 
second color plane, the interpolation of a second color value at the given pixel at 
pixel coordinates (x,y) performed according to: 

i d ( x > y) = (*> y)i c 0> y) + s d , c (*» y) 

where l d is the second color value, l c is either the green sampled value or the green 
interpolated value at pixel coordinates (x,y), Sd, c is a slowly varying function of a shift 
in color c, and td fC is a slowly varying function that can change the intensity gradient. 

33. Apparatus for processing a mosaic image, the apparatus comprising: 
means for using sampled values in a color plane of the mosaic image to 

determine the likelihoods of a given pixel belong to the same region as at least two 
different neighbors lying in different directions; and 

means for using the likelihoods and sampled values in the different directions 
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to interpolate a value at the given pixel, the interpolated value being in the color 
plane. 

34. The apparatus of claim 33, further comprising means for aquiring the 
mosaic image. 

35. A digital camera comprising: 

a photosensor array for acquiring mosaic images; and 
a digital signal processor for processing the mosaic images, the processing 
including interpolating missing values from sampled values in the mosaic image, the 
interpolation of a given pixel in a given color plane including determining the 
likelihoods of neighbors belonging to the same region as the given pixel, the 
neighbors being in at least two different directions relative to the given pixel; and 
using the likelihoods and the sampled values of the neighbors to interpolate the given 
color at the given pixel. 

36. The digital camera of claim 35, wherein the processing includes using 
sampled values of a second color to compute terms for correcting the sampled 
values of the given color. 

37. An article for instructing a processor to process a mosaic image, the 
article comprising memory encoded with instructions for instructing the processor to 
interpolate missing values of a first color, the interpolation of the first color of a given 
pixel including determining likelihoods of neighbors having sampled values of the first 
color being in the same region as the given pixel, and using the sampled values and 
the likelihoods of those neighbors to interpolate the first color at the given pixel, at 
least one neighbor being in a first direction relative to the given pixel, at least one 
neighbor being in a second direction relative to the given pixel, where the first and 
second directions are orthogonal. 
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38. The article of claim 37, wherein the processing includes using sampled 
values of a second color to compute terms for correcting the sampled values of the 
first color. 
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